New UI step 1 #68

benoit74 · 2023-10-16T08:15:43Z

This is a reopen of #65 with an intermediate base branch new_ui since know limitation below prevent us to merge tomain

Changes:

Scraper (Python code) has been moved to the scraper subfolder
Vue.JS is now used as main UI framework ; all its code is in the zimui subfolder ; it is rendered with Vite to produce a static website
QA and Tests workflows have been adapted
- to the new folder structure
- to also QA and Test the Vue.JS part
precommit hooks have been configured for the Vue.JS part
Dockerfile has been adapted to first build the Vue.JS part in a dedicated stage and then embed the generated files into the final Python-based image
Topics are stored as JSON files in the ZIM
- JSON is generated by pydantic
- these files are consumed by the Vue.JS UI
- content (video, audio, pdf, epub, ...) is still rendered by Jinja2 as before
URLs are meaningful slugs
- generated by Python slugify lib
- from Kolibri node title
- should two distinct nodes have the same title resulting in the same slug, conflicts are handled with a _1, _2, ... suffix
changes in the ZIM "folder" structure:
- files generated by Vite are placed in /
- thumbnails are placed in /thumbnails
- JSON files generated to render topics are placed in /topics
- most Kolibri content (video, audio, ePub, PDF) are placed in /files (some content is still placed at the root to not break some stuff which was found hard to fix for now, will be tackled in specific issues for each content type)
legacy MANIFEST.in has been deleted (left-over from migration to hatch)
is_front property has been adjusted when adding the item to the ZIM
one new CLI argument --zimui-dist to specify the folder where zimui has been built (by Vite)

Known side effects to be discussed:

topics are not searchable anymore (will be fixed by Better UI: add search for topics back #67)
navigation breadcrumbs are not here anymore (will be fixed by Better UI: add navigation breadcrumbs back #64)

Jaifroid · 2023-10-16T12:46:45Z

I realize this relates to the previous PR, but I suppose it will carry over to this one. I noticed that there are errors in the quizzes contained in small_test_new_ui_2023-10.zim. I wondered whether you also see these in other apps, or whether it might be an issue with the Kiwix JS software. See screenshot below. The quiz still works, so it's only cosmetic, but it's unusual that missing stylesheets can be displayed as errors in the UI:

rgaudin · 2023-10-16T15:25:10Z

  * most Kolibri content (video, audio, ePub, PDF) are placed in `/static` (some content is still placed at the root to not break some stuff which was found hard to fix for now, will be tackled in specific issues for each content type)

#65 (comment)

rgaudin · 2023-10-16T15:26:10Z

I've just uploaded test ZIMs to dev library (please forgive the filename, I suspect they are not 100% compatible with our naming convention).

minimal test :
- a very small channel I created for test purpose, with all kind of resources
- https://dev.library.kiwix.org/viewer#minimal_test_new_ui_2023-10
small test :
- a small channel I created for test purpose, with nested nodes and various kinds of resources
- https://dev.library.kiwix.org/viewer#small_test_new_ui_2023-10
african story book :
- the "official" one
- not the best test case because there are no thumbnails in the channel, so it does not look very good (I opened Enhance African Storybooks content zim-requests#729 to speak about it btw)
- new UI: https://dev.library.kiwix.org/viewer#africanstorybook.org_mul_all_newui_2023-10
- old UI: https://library.kiwix.org/viewer#africanstorybook.org_mul_all_2023-10

Jaifroid · 2023-10-16T15:28:11Z

  * most Kolibri content (video, audio, ePub, PDF) are placed in `/static` (some content is still placed at the root to not break some stuff which was found hard to fix for now, will be tackled in specific issues for each content type)

#65 (comment)

Ah, OK, I hadn't associated that comment with this error. Thanks.

rgaudin · 2023-10-16T15:40:04Z

in ASB, frequently (not systematically), when I browse one language and come back, I find myself at the bottom of the page (seeing Yoruba), while I left homepage on top of the page. This is disturbing.

I find those arrows strange ; are those from the Endless UI? Can you share the endless UI link again? Would help compare.

assets/perseus/index.html is apparently is_front since it's served by random()

When I was testing I encountered many times a broken image on the first page of PDF (the thumbnail) and then it worked and I can't reproduce. In case you're seen it as well, there might be an issue somewhere ; if you did not, we'll have to wait for it to resurface.

benoit74 · 2023-10-17T06:48:24Z

@Jaifroid thank you for this report ; it is not normal and will be fixed. Maybe this is displayed in the page due to perseus codebase which handle loading these assets very nicely? (I don't know tbh)

@rgaudin thank you for these first feedbacks:

I did not encountered the broken image on the first page of the PDF, I will check on a browser with a cleared cache
I will fix as well the "bottom of the page" issue
Your screen is definitely too big, I will buy you a smaller one 🤣 ; I knew the arrows where not perfect, but they are definitely bad on such a big screen ; I will fix this as well ; original endless UI (and link to it) is in the issue Better UI: minimal first step #53

rgaudin

Thank you. Can you also update the changelog? It's part of our process to update it along changes.

.gitignore

README.md

scraper/pyproject.toml

scraper/src/kolibri2zim/schemas.py

zimui/src/components/TopicSection.vue

zimui/src/components/TopicCard.vue

benoit74 · 2023-10-20T12:55:05Z

I think I've sorted out all comments.

I've added issues I consider mandatory before merging this base branch to the 1.2.0 milestone. Feel free to speak up if there is something missing (or something not mandatory).

benoit74 · 2023-10-20T13:01:33Z

And I've removed the "Fix #xxx" so that issue #53 is not closed until all other issues are closed, and #31 is not closed until decision is made regarding slugs computation.

rgaudin

Thank you ; I believe it's good to go 👍

The CamelCase change is unexpected in-between reviews. Please mention such changes in your comments next time.

Can you also open a ticket on your repo of choice to implement get_web_deps in Python?

rgaudin · 2023-10-21T13:06:55Z

zimui/src/stores/main.ts

+          this.channelData = response.data as Channel
+        },
+        (_) => {
+          this.isLoading = false


Maybe log the error in the console?

This will be tackled more appropriately by #70

rgaudin · 2023-10-21T13:07:03Z

zimui/src/stores/main.ts

+          return response.data as Topic
+        },
+        (_) => {
+          this.isLoading = false


Same also 😀

rgaudin · 2023-10-21T13:09:10Z

zimui/src/components/TopicSection.vue

-const limitCardsPerSections = (inputArray: any[], section_slug: string) => {
+/**
+ * Keep only the 10 first items of the input array. If more than 10 items
+ * where present in the list, a "magic/virtual" 11th item is created to indicate


typo “were”

rgaudin · 2023-10-21T13:11:42Z

zimui/src/components/TopicSection.vue

+): TopicCardData[][] => {
+  return cards.reduce(
+    (all: TopicCardData[][], one: TopicCardData, i: number) => {
+      const ch = Math.floor(i / cardsPerChunk)


I think we can afford words for variable name: chunk, index. Even all is vague

kelson42 · 2023-10-21T15:09:46Z

@benoit74 I have copied the two tickets fixed by previous version of this PR in the description. Hope this is OK.

kelson42 · 2023-10-21T15:31:39Z

@benoit74 I have finally check the new ZIM, here my remarks:

We have a clear improvement, even if I believe this is still the very beginning.
EPUB filename should be meaningfull. For example qaxalefi-obboleewwan-isaa.epub at https://dev.library.kiwix.org/viewer#africanstorybook.org_mul_all_newui_2023-10/static/qaxalefi-obboleewwan-isaa
I don't really understand the role of the one tab "Main document (EPUB)". It does not make sense to have a tab system if there only one tab + "Main document (EPUB)" does not bring anything useful for the user. I would simply remove things and just keep the button to download the EPUB. I kind of suspect that we might have situation where we have many tabs... but here this is not the case in the whole ZIM.
Random button seems broken and returns alwasy https://dev.library.kiwix.org/viewer#africanstorybook.org_mul_all_newui_2023-10/assets/perseus/index.html. I believe the random (and the suggestions) should work only on the stories and categories.
Suggestions not working at all looks like.
The interesting part of a page with a story is the book (EPUB renderer), but the book itself is pretty small an takes only a very small portion of the screen (with big margins everywhere). I suspect the book should be displayed bigger.
We definitely have a problem with everywhere the same thumbnail for categories
I wonder if the button "OPEN" on thumbnail makes sense at all, but I question its behaviour if the whole content is in French. Localisation (even if still not existing) should be taken in consideration.
Similar as for youtube, we should find a way - automatically or not - to get ride of this basis white/grey background colors.

benoit74 · 2023-10-22T14:59:42Z

@benoit74 I have copied the two tickets fixed by previous version of this PR in the description. Hope this is OK.

No it's not, see my last comment #68 (comment)

We have a clear improvement, even if I believe this is still the very beginning.

Thank you. And yes, this is the very beginning, this is why we have so many new issues to fix, all titled "Better UI"

EPUB filename should be meaningfull. For example qaxalefi-obboleewwan-isaa.epub at https://dev.library.kiwix.org/viewer#africanstorybook.org_mul_all_newui_2023-10/static/qaxalefi-obboleewwan-isaa

Will be covered by #55 ; same for all other kind of assets in fact

I don't really understand the role of the one tab "Main document (EPUB)". It does not make sense to have a tab system if there only one tab + "Main document (EPUB)" does not bring anything useful for the user. I would simply remove things and just keep the button to download the EPUB. I kind of suspect that we might have situation where we have many tabs... but here this is not the case in the whole ZIM.

Idem, this is the old design

Random button seems broken and returns alwasy https://dev.library.kiwix.org/viewer#africanstorybook.org_mul_all_newui_2023-10/assets/perseus/index.html. I believe the random (and the suggestions) should work only on the stories and categories.

Will be covered by #67

Suggestions not working at all looks like.

Idem, they are working if you search for a PDF / ePub title AFAIK

The interesting part of a page with a story is the book (EPUB renderer), but the book itself is pretty small an takes only a very small portion of the screen (with big margins everywhere). I suspect the book should be displayed bigger.

This is the old design

We definitely have a problem with everywhere the same thumbnail for categories

This is why I am asking for a rework of Kolibri channel for African Storybook, once this is fixed we will have proper thumbnails. These thumbnails are supposed to be here and are handled if present, I don't want to create them on the scraper side.

I wonder if the button "OPEN" on thumbnail makes sense at all, but I question its behaviour if the whole content is in French.

This is the original Endless design

Localisation (even if still not existing) should be taken in consideration.

There is no issue (need ?) for this so far AFAIK

Similar as for youtube, we should find a way - automatically or not - to get ride of this basis white/grey background colors.

I don't get what you are speaking about

benoit74 · 2023-10-22T15:38:54Z

@rgaudin I think I've fixed your last comment, let merge if this is ok for you.

rgaudin · 2023-10-22T21:27:03Z

@rgaudin I think I've fixed your last comment, let merge if this is ok for you.

It is, thank you

benoit74 added 13 commits October 2, 2023 13:16

Move scraper to a subfolder

317dadb

Minimal Vue.js-based UI pushed to the ZIM

01612ee

Fix UI bug when rendering an ePub with a hack

c543c9d

Fix format + lint issues

bee8a15

Add prettier and eslint for zimui

8909dda

Add zimui QA and Tests

c925338

Fix README for zimui / Vue.JS adaptation

e3c65b1

Fix CI

aa720ae

Remove zimui tests since there is none for now

ae47164

Documentation + cleanup

9d8803c

Do not warn on explicit any in JS

d1e16c8

Rename 'static' folder to 'files'

28cdd9f

Disable 'no-explicit-any' one line at a time

3e986ba

benoit74 self-assigned this Oct 16, 2023

benoit74 marked this pull request as ready for review October 16, 2023 08:39

benoit74 requested a review from rgaudin October 16, 2023 08:39

rgaudin requested changes Oct 18, 2023

View reviewed changes

benoit74 mentioned this pull request Oct 19, 2023

URLs should be meaningful #31

Closed

benoit74 force-pushed the new_ui_step_1 branch from e662bed to c6a22f6 Compare October 20, 2023 09:43

benoit74 added 5 commits October 20, 2023 13:13

Many fixes following review

fffaefe

Fix is_front for assets and some other resources

4dd228c

Update CHANGELOG

d63cabd

Remove fonts from repo

6bea129

Remove ugly index.html main page + add is_front to this page

afabde7

Empty main path is not working with Vue.JS

96440af

benoit74 force-pushed the new_ui_step_1 branch from c6a22f6 to 96440af Compare October 20, 2023 12:16

Add missing footer

039294c

benoit74 requested a review from rgaudin October 20, 2023 12:55

rgaudin approved these changes Oct 21, 2023

View reviewed changes

Typo fix + simplify code + add meaningful variable names

56dcad2

benoit74 merged commit de74cee into new_ui Oct 23, 2023
6 checks passed

benoit74 deleted the new_ui_step_1 branch October 23, 2023 06:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New UI step 1 #68

New UI step 1 #68

benoit74 commented Oct 16, 2023 •

edited

Loading

Jaifroid commented Oct 16, 2023

rgaudin commented Oct 16, 2023

rgaudin commented Oct 16, 2023

Jaifroid commented Oct 16, 2023

rgaudin commented Oct 16, 2023

benoit74 commented Oct 17, 2023

rgaudin left a comment

benoit74 commented Oct 20, 2023

benoit74 commented Oct 20, 2023

rgaudin left a comment

rgaudin Oct 21, 2023

benoit74 Oct 22, 2023

rgaudin Oct 21, 2023

benoit74 Oct 22, 2023

rgaudin Oct 21, 2023

rgaudin Oct 21, 2023

benoit74 Oct 22, 2023

kelson42 commented Oct 21, 2023

kelson42 commented Oct 21, 2023

benoit74 commented Oct 22, 2023 •

edited

Loading

benoit74 commented Oct 22, 2023

rgaudin commented Oct 22, 2023

New UI step 1 #68

New UI step 1 #68

Conversation

benoit74 commented Oct 16, 2023 • edited Loading

Jaifroid commented Oct 16, 2023

rgaudin commented Oct 16, 2023

rgaudin commented Oct 16, 2023

Jaifroid commented Oct 16, 2023

rgaudin commented Oct 16, 2023

benoit74 commented Oct 17, 2023

rgaudin left a comment

Choose a reason for hiding this comment

benoit74 commented Oct 20, 2023

benoit74 commented Oct 20, 2023

rgaudin left a comment

Choose a reason for hiding this comment

rgaudin Oct 21, 2023

Choose a reason for hiding this comment

benoit74 Oct 22, 2023

Choose a reason for hiding this comment

rgaudin Oct 21, 2023

Choose a reason for hiding this comment

benoit74 Oct 22, 2023

Choose a reason for hiding this comment

rgaudin Oct 21, 2023

Choose a reason for hiding this comment

rgaudin Oct 21, 2023

Choose a reason for hiding this comment

benoit74 Oct 22, 2023

Choose a reason for hiding this comment

kelson42 commented Oct 21, 2023

kelson42 commented Oct 21, 2023

benoit74 commented Oct 22, 2023 • edited Loading

benoit74 commented Oct 22, 2023

rgaudin commented Oct 22, 2023

benoit74 commented Oct 16, 2023 •

edited

Loading

benoit74 commented Oct 22, 2023 •

edited

Loading